Inverse reinforcement learning to control a robotic arm using a Brain-Computer Interface

The goal of this project is to use inverse reinforce- ment learning to better control a JACO robotic arm developed by Kinova in a Brain-Computer Interface (BCI). A self-paced BCI such as a motor imagery based-BCI allows the subject to give orders at any time to freely control a device. But using this paradigm, even after a long training, the accuracy of the classifier used to recognize the order is not 100%. While a lot of studies try to improve the accuracy using a preprocessing stage that improves the feature extraction, we work on a post- processing solution. The classifier used to recognize the mental commands will provide as outputs a value for each command such as the posterior probability. But the executed action will not only depend on this information. A decision process will also take into account the position of the robotic arm and previous trajectories. More precisely, the decision process will be obtained applying an inverse reinforcement learning (IRL) on a subset of trajectories specified by an expert. At the end of the workshop, the convergence of the inverse reinforcement algorithm has not been achieved. Nevertheless, we developed a whole processing chain based on OpenViBE for controlling 2D- movements and we present how to deal with this high dimensional time series problem with a lot of noise which is unusual for the IRL community

Apprentissage par imitation dans un cadre batch, off-policy et sans modèle

National audienceCe papier traite le problème de l'apprentissage par imitation, c'est à dire la résolution du problème du contrôle optimal à partir de données tirées d'une démonstration d'expert. L'apprentissage par renforcement inverse (IRL) propose un cadre efficace pour résoudre ce problème. En se basant sur l'hypothèse que l'expert maximise un critère, l'IRL essaie d'apprendre la récompense qui définit ce critère à partir de trajectoires d'exemple. Beaucoup d'algorithmes d'IRL font l'hypothèse de l'existence d'un bon approximateur linéaire pour la fonction de récompense et calculent l'attribut moyen (le cumul moyen pondéré des fonctions de base, relatives à la paramétrisation linéaire supposée de la récompense, évaluées en les états d'une trajectoire associée à une certaine politique) via une estimation de Monte-Carlo. Cela implique d'avoir accès à des trajectoires complète de l'expert ainsi qu'à au moins un modèle génératif pour tester les politiques intermédiaires. Dans ce papier nous introduisons une méthode de différence temporelle, LSTD-µ, pour calculer cet attribut moyen. Cela permet d'étendre l'apprentissage par imitation aux cas batch et off-policy

FM-AFM with hanging fiber probe for the study of liquid / liquid interfaces

This article describes how an FM-AFM using a hanging fiber force probe made from a quartz tuning fork provides local measurements on liquid-liquid interfaces. After detailing the manufacture and calibration of the force probe, we provide evidence that this AFM is suitable for quantitative measurements at the interface between two liquids. The repeatability of the measurements allows a PDMS / water moving interface to be monitored over several hours : the evaporation of a water droplet immersed in PDMS is observed, and its interfacial tension evolution over time is measured. Deformation of the interface is also observed. These capabilities, and preliminary results on the interface between two immiscible liquids, pave the way for interface manipulation and study of complex fluid-fluid interfaces

A cascaded supervised learning approach to inverse reinforcement learning

International audienceThis paper considers the Inverse Reinforcement Learning (IRL) problem, that is inferring a reward function for which a demonstrated expert policy is optimal. We propose to break the IRL problem down into two generic Supervised Learning steps: this is the Cascaded Supervised IRL (CSI) approach. A classification step that defines a score function is followed by a regression step providing a reward function. A theoretical analysis shows that the demonstrated expert policy is nearoptimal for the computed reward function. Not needing to repeatedly solve a Markov Decision Process (MDP) and the ability to leverage existing techniques for classification and regression are two important advantages of the CSI approach. It is furthermore empirically demonstrated to compare positively to state-of-the-art approaches when using only transitions sampled according to the expert policy, up to the use of some heuristics. This is exemplified on two classical benchmarks (the mountain car problem and a highway driving simulator)

Automated assessment and tracking of human body thermal variations using unsupervised clustering

The presented approach addresses a review of the overheating that occurs during radiological examinations, such as magnetic resonance imaging, and a series of thermal experiments to determine a thermally suitable fabric material that should be used for radiological gowns. Moreover, an automatic system for detecting and tracking of the thermal fluctuation is presented. It applies hue-saturated-value-based kernelled k-means clustering, which initializes and controls the points that lie on the region-of-interest (ROI) boundary. Afterward, a particle filter tracks the targeted ROI during the video sequence independently of previous locations of overheating spots. The proposed approach was tested during experiments and under conditions very similar to those used during real radiology exams. Six subjects have voluntarily participated in these experiments. To simulate the hot spots occurring during radiology, a controllable heat source was utilized near the subject’s body. The results indicate promising accuracy for the proposed approach to track hot spots. Some approximations were used regarding the transmittance of the atmosphere, and emissivity of the fabric could be neglected because of the independence of the proposed approach for these parameters. The approach can track the heating spots continuously and correctly, even for moving subjects, and provides considerable robustness against motion artifact, which occurs during most medical radiology procedures

Incremental low rank noise reduction for robust infrared tracking of body temperature during medical imaging

Thermal imagery for monitoring of body temperature provides a powerful tool to decrease health risks (e.g., burning) for patients during medical imaging (e.g., magnetic resonance imaging). The presented approach discusses an experiment to simulate radiology conditions with infrared imaging along with an automatic thermal monitoring/tracking system. The thermal tracking system uses an incremental low-rank noise reduction applying incremental singular value decomposition (SVD) and applies color based clustering for initialization of the region of interest (ROI) boundary. Then a particle filter tracks the ROI(s) from the entire thermal stream (video sequence). The thermal database contains 15 subjects in two positions (i.e., sitting, and lying) in front of thermal camera. This dataset is created to verify the robustness of our method with respect to motion-artifacts and in presence of additive noise (2–20%—salt and pepper noise). The proposed approach was tested for the infrared images in the dataset and was able to successfully measure and track the ROI continuously (100% detecting and tracking the temperature of participants), and provided considerable robustness against noise (unchanged accuracy even in 20% additive noise), which shows promising performanc

Infrared vision for artwork and cultural heritage NDE studies: principles and case studies

This text briefly presents the basis of 'infrared vision' in the context of cultural heritage studies. Infrared vision here encompasses near-infrared as well as thermal infrared schemes of inspection. The theory is briefly presented and attention is then focused on several non-destructive evaluation (NDE) case studies in cultural heritage: painting artwork, under-painting lettering retrieval and the investigation of Egyptian pyramids through the ScanPyramids Mission, led by the Faculty of Engineering of Cairo University and the HIP (Heritage Innovation Preservation) Institute

Comparative study of Line Scan and Flying Line Active IR Thermography operated with a 6-axis robot

International audienceIn this paper, two Non Destructive Testing approaches by active infrared thermography mounted on a 6-axis robot are presented and studied. Data acquisition and thermal excitation is carried out dynamically over various CFRP specimens with increasing geometry complexity, from planar, to convex and concave shapes. An automated procedure is proposed to reconstruct thermal image sequences issued from the two scanning procedure studied: Line Scan and Flying Line procedures. Defective area detection is performed by image processing and an inverse technique based on thermal quadrupole method is used to map the depth of flaws. Results obtained are discussed and perspectives are addressed

EBEX: A balloon-borne CMB polarization experiment

EBEX is a NASA-funded balloon-borne experiment designed to measure the polarization of the cosmic microwave background (CMB). Observations will be made using 1432 transition edge sensor (TES) bolometric detectors read out with frequency multiplexed SQuIDs. EBEX will observe in three frequency bands centered at 150, 250, and 410 GHz, with 768, 384, and 280 detectors in each band, respectively. This broad frequency coverage is designed to provide valuable information about polarized foreground signals from dust. The polarized sky signals will be modulated with an achromatic half wave plate (AHWP) rotating on a superconducting magnetic bearing (SMB) and analyzed with a fixed wire grid polarizer. EBEX will observe a patch covering ~1% of the sky with 8' resolution, allowing for observation of the angular power spectrum from \ell = 20 to 1000. This will allow EBEX to search for both the primordial B-mode signal predicted by inflation and the anticipated lensing B-mode signal. Calculations to predict EBEX constraints on r using expected noise levels show that, for a likelihood centered around zero and with negligible foregrounds, 99% of the area falls below r = 0.035. This value increases by a factor of 1.6 after a process of foreground subtraction. This estimate does not include systematic uncertainties. An engineering flight was launched in June, 2009, from Ft. Sumner, NM, and the long duration science flight in Antarctica is planned for 2011. These proceedings describe the EBEX instrument and the North American engineering flight.Comment: 12 pages, 9 figures, Conference proceedings for SPIE Millimeter, Submillimeter, and Far-Infrared Detectors and Instrumentation for Astronomy V (2010